Averaging for some simple constrained Markov processes

Some simple but challenging Markov processes

In this note, we present few examples of Piecewise Deterministic Markov Processes and their long time behavior. They share two important features: they are related to concrete models (in biology, networks, chemistry,. . . ) and they are mathematically rich. Their mathematical study relies on coupling method, spectral decomposition, PDE technics, functional inequalities. We also relate these sim...

Constrained Markov Decision Processes

2 i To Tania and Einat ii Preface In many situations in the optimization of dynamic systems, a single utility for the optimizer might not suuce to describe the real objectives involved in the sequential decision making. A natural approach for handling such cases is that of optimization of one objective with constraints on other ones. This allows in particular to understand the tradeoo between t...

Relaxation for Constrained Decentralized Markov Decision Processes

This paper studies a class of decentralized multi-agent stochastic optimization problems. In these problems, each agent has only a partial view of the world state, and a partial control of the actions but must cooperatively maximize the long-term system reward. The state that an agent observe consists of two parts a common public component and an agentspecific private component. Importantly, ta...

Optimal Stopping Problems for Some Markov Processes

In this paper, we solve explicitly the optimal stopping problem with random discounting and an additive functional as cost of observations for a regular linear diffusion. We also extend the results to the class of one-sided regular Feller processes. This generalizes the result of Beibel and Lerche [Statist. Sinica 7 (1997) 93–108] and [Teor. Veroyatn. Primen. 45 (2000) 657–669] and Irles and Pa...

Some Peculiar Semi-markov Processes